Methods and apparatus for analyzing specificity in clinical documentation

ABSTRACT

A set of one or more clinical facts may be collected from a clinician&#39;s encounter with a patient. From the set of facts, it may be determined that an additional fact that provides additional specificity to the set of facts may possibly be ascertained from the patient encounter. A user may be alerted that the additional fact may possibly be ascertained from the patient encounter.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/030,974, entitled “METHODS AND APPARATUS FOR ANALYZING SPECIFICITY INCLINICAL DOCUMENTATION,” filed on Feb. 18, 2011, which is incorporatedherein by reference in its entirety.

BACKGROUND OF INVENTION

1. Field of Invention

The techniques described herein are directed generally to the field ofclinical documentation, and more particularly to techniques for thecreation and use of patient records in clinical settings.

2. Description of the Related Art

Clinical documentation is an important process in the healthcareindustry. Most healthcare institutions maintain a longitudinal medicalrecord (e.g., spanning multiple observations or treatments over time)for each of their patients, documenting the patient's history,encounters with clinical staff within the institution, treatmentreceived, and plans for future treatment. Such documentation facilitatesmaintaining continuity of care for the patient across multipleencounters with various clinicians over time. In addition, when aninstitution's medical records for large numbers of patients areconsidered in the aggregate, the information contained therein can beuseful for educating clinicians as to treatment efficacy and bestpractices, for internal auditing within the institution, for qualityassurance, etc.

Historically, each patient's medical record was maintained as a physicalpaper folder, often referred to as a “medical chart”, or “chart”. Eachpatient's chart would include a stack of paper reports, such as intakeforms, history and immunization records, laboratory results andclinicians' notes. Following an encounter with the patient, such as anoffice visit, a hospital round or a surgical procedure, the clinicianconducting the encounter would provide a narrative note about theencounter to be included in the patient's chart. Such a note couldinclude, for example, a description of the reason(s) for the patientencounter, an account of any vital signs, test results and/or otherclinical data collected during the encounter, one or more diagnosesdetermined by the clinician from the encounter, and a description of aplan for further treatment. Often, the clinician would verbally dictatethe note into an audio recording device or a telephone giving access tosuch a recording device, to spare the clinician the time it would taketo prepare the note in written form. Later, a medical transcriptionistwould listen to the audio recording and transcribe it into a textdocument, which would be inserted on a piece of paper into the patient'schart for later reference.

Currently, many healthcare institutions are transitioning or havetransitioned from paper documentation to electronic medical recordsystems, in which patients' longitudinal medical information is storedin a data repository in electronic form. Besides the significantphysical space savings afforded by the replacement of paperrecord-keeping with electronic storage methods, the use of electronicmedical records also provides beneficial time savings and otheropportunities to clinicians and other healthcare personnel. For example,when updating a patient's electronic medical record to reflect a currentpatient encounter, a clinician need only document the new informationobtained from the encounter, and need not spend time entering unchangedinformation such as the patient's age, gender, medical history, etc.Electronic medical records can also be shared, accessed and updated bymultiple different personnel from local and remote locations throughsuitable user interfaces and network connections, eliminating the needto retrieve and deliver paper files from a crowded file room.

SUMMARY OF INVENTION

One embodiment is directed to a method comprising: receiving an originaltext that is a representation of a narration of a patient encounterprovided by a clinician; re-formatting the original text, using at leastone processor, to produce a formatted text; extracting one or moreclinical facts from the formatted text, wherein a first fact of the oneor more clinical facts is extracted from a first portion of theformatted text, wherein the first portion of the formatted text is aformatted version of a first portion of the original text; andmaintaining a linkage between the first fact and the first portion ofthe original text.

Another embodiment is directed to apparatus comprising at least oneprocessor, and a memory storing processor-executable instructions that,when executed by the at least one processor, perform a methodcomprising: receiving an original text that is a representation of anarration of a patient encounter provided by a clinician; re-formattingthe original text to produce a formatted text; extracting one or moreclinical facts from the formatted text, wherein a first fact of the oneor more clinical facts is extracted from a first portion of theformatted text, wherein the first portion of the formatted text is aformatted version of a first portion of the original text; andmaintaining a linkage between the first fact and the first portion ofthe original text.

Another embodiment is directed to at least one computer-readable storagemedium encoded with a plurality of computer-executable instructionsthat, when executed, perform a method comprising: receiving an originaltext that is a representation of a narration of a patient encounterprovided by a clinician; re-formatting the original text to produce aformatted text; extracting one or more clinical facts from the formattedtext, wherein a first fact of the one or more clinical facts isextracted from a first portion of the formatted text, wherein the firstportion of the formatted text is a formatted version of a first portionof the original text; and maintaining a linkage between the first factand the first portion of the original text.

Another embodiment is directed to a method comprising: extracting, usingat least one processor, a plurality of clinical facts from a free-formnarration of a patient encounter provided by a clinician, wherein theplurality of clinical facts comprises a first fact and a second fact,wherein the first fact is extracted from a first portion of thefree-form narration, and wherein the second fact is extracted from asecond portion of the free-form narration; and providing to a user afirst indicator that indicates a first linkage between the first factand the first portion of the free-form narration, and a secondindicator, different from the first indicator, that indicates a secondlinkage between the second fact and the second portion of the free-formnarration.

Another embodiment is directed to apparatus comprising at least oneprocessor, and a memory storing processor-executable instructions that,when executed by the at least one processor, perform a methodcomprising: extracting a plurality of clinical facts from a free-formnarration of a patient encounter provided by a clinician, wherein theplurality of clinical facts comprises a first fact and a second fact,wherein the first fact is extracted from a first portion of thefree-form narration, and wherein the second fact is extracted from asecond portion of the free-form narration; and providing to a user afirst indicator that indicates a first linkage between the first factand the first portion of the free-form narration, and a secondindicator, different from the first indicator, that indicates a secondlinkage between the second fact and the second portion of the free-formnarration.

Another embodiment is directed to at least one computer-readable storagemedium encoded with a plurality of computer-executable instructionsthat, when executed, perform a method comprising: extracting a pluralityof clinical facts from a free-form narration of a patient encounterprovided by a clinician, wherein the plurality of clinical factscomprises a first fact and a second fact, wherein the first fact isextracted from a first portion of the free-form narration, and whereinthe second fact is extracted from a second portion of the free-formnarration; and providing to a user a first indicator that indicates afirst linkage between the first fact and the first portion of thefree-form narration, and a second indicator, different from the firstindicator, that indicates a second linkage between the second fact andthe second portion of the free-form narration.

Another embodiment is directed to a method comprising: collecting a setof one or more clinical facts from a clinician's encounter with apatient; determining from the set of facts, using at least oneprocessor, that an additional fact that provides additional specificityto the set of facts may possibly be ascertained from the patientencounter; and alerting a user that the additional fact may possibly beascertained from the patient encounter.

Another embodiment is directed to apparatus comprising at least oneprocessor, and a memory storing processor-executable instructions that,when executed by the at least one processor, perform a methodcomprising: collecting a set of one or more clinical facts from aclinician's encounter with a patient; determining from the set of factsthat an additional fact that provides additional specificity to the setof facts may possibly be ascertained from the patient encounter; andalerting a user that the additional fact may possibly be ascertainedfrom the patient encounter.

Another embodiment is directed to at least one computer-readable storagemedium encoded with a plurality of computer-executable instructionsthat, when executed, perform a method comprising: collecting a set ofone or more clinical facts from a clinician's encounter with a patient;determining from the set of facts that an additional fact that providesadditional specificity to the set of facts may possibly be ascertainedfrom the patient encounter; and alerting a user that the additional factmay possibly be ascertained from the patient encounter.

Another embodiment is directed to a method comprising: collecting a setof one or more clinical facts from a clinician's encounter with apatient; determining, using at least one processor, that an unspecifieddiagnosis not included in the set of facts may possibly be ascertainedfrom the patient encounter; and alerting a user that the unspecifieddiagnosis may possibly be ascertained from the patient encounter.

Another embodiment is directed to apparatus comprising at least oneprocessor, and a memory storing processor-executable instructions that,when executed by the at least one processor, perform a methodcomprising: collecting a set of one or more clinical facts from aclinician's encounter with a patient; determining that an unspecifieddiagnosis not included in the set of facts may possibly be ascertainedfrom the patient encounter; and alerting a user that the unspecifieddiagnosis may possibly be ascertained from the patient encounter.

Another embodiment is directed to at least one computer-readable storagemedium encoded with a plurality of computer-executable instructionsthat, when executed, perform a method comprising: collecting a set ofone or more clinical facts from a clinician's encounter with a patient;determining that an unspecified diagnosis not included in the set offacts may possibly be ascertained from the patient encounter; andalerting a user that the unspecified diagnosis may possibly beascertained from the patient encounter.

Another embodiment is directed to a method comprising: determining,based on a free-form narration of a patient encounter provided by aclinician, that one or more clinical facts could possibly be ascertainedfrom the patient encounter; providing to a user one or more optionscorresponding to the one or more clinical facts; receiving from the usera selection of a first option of the one or more options, the firstoption corresponding to a first fact of the one or more clinical facts;and updating a textual representation of the free-form narration, usingat least one processor, to identify the first fact as having beenascertained from the patient encounter.

Another embodiment is directed to apparatus comprising at least oneprocessor, and a memory storing processor-executable instructions that,when executed by the at least one processor, perform a methodcomprising: determining, based on a free-form narration of a patientencounter provided by a clinician, that one or more clinical facts couldpossibly be ascertained from the patient encounter; providing to a userone or more options corresponding to the one or more clinical facts;receiving from the user a selection of a first option of the one or moreoptions, the first option corresponding to a first fact of the one ormore clinical facts; and updating a textual representation of thefree-form narration to identify the first fact as having beenascertained from the patient encounter.

Another embodiment is directed to at least one computer-readable storagemedium encoded with a plurality of computer-executable instructionsthat, when executed, perform a method comprising: determining, based ona free-form narration of a patient encounter provided by a clinician,that one or more clinical facts could possibly be ascertained from thepatient encounter; providing to a user one or more options correspondingto the one or more clinical facts; receiving from the user a selectionof a first option of the one or more options, the first optioncorresponding to a first fact of the one or more clinical facts; andupdating a textual representation of the free-form narration to identifythe first fact as having been ascertained from the patient encounter.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings are not intended to be drawn to scale. In thedrawings, each identical or nearly identical component that isillustrated in various figures is represented by a like numeral. Forpurposes of clarity, not every component may be labeled in everydrawing. In the drawings:

FIG. 1 is a block diagram of an exemplary operating environment for asystem in accordance with some embodiments of the present invention;

FIG. 2 is a screenshot illustrating an exemplary graphical userinterface for a clinical fact review system in accordance with someembodiments of the present invention;

FIGS. 3A and 3B are screenshots illustrating an exemplary display ofclinical facts in a user interface in accordance with some embodimentsof the present invention;

FIG. 4 is a screenshot illustrating an exemplary display of linkagebetween text and a clinical fact in accordance with some embodiments ofthe present invention;

FIG. 5 is a screenshot illustrating an exemplary interface for enteringa clinical fact in accordance with some embodiments of the presentinvention;

FIG. 6 is a flowchart illustrating an exemplary method for formattingtext for clinical fact extraction in accordance with some embodiments ofthe present invention;

FIG. 7 is a flowchart illustrating an exemplary method for linkingextracted clinical facts to text in accordance with some embodiments ofthe present invention;

FIG. 8 is a flowchart illustrating an exemplary method for analyzingspecificity in accordance with some embodiments of the presentinvention;

FIG. 9 is a flowchart illustrating an exemplary method for identifyingan unspecified diagnosis in accordance with some embodiments of thepresent invention;

FIG. 10 is a flowchart illustrating an exemplary method for updatingtext in accordance with some embodiments of the present invention; and

FIG. 11 is a block diagram of an exemplary computer system on whichaspects of the present invention may be implemented.

DETAILED DESCRIPTION

An Electronic Health Record (EHR) is an electronic medical record thatgenerally is maintained by a specific healthcare institution andcontains data documenting the care that a specific patient has receivedfrom that institution over time. Typically, an EHR is maintained as astructured data representation, such as a database with structuredfields. Each piece of information stored in such an EHR is typicallyrepresented as a discrete (e.g., separate) data item occupying a fieldof the EHR database. For example, a 55-year old male patient named JohnDoe may have an EHR database record with “John Doe” stored in thepatient_name field, “55” stored in the patient_age field, and “Male”stored in the patient_gender field. Data items or fields in such an EHRare structured in the sense that only a certain limited set of validinputs is allowed for each field. For example, the patient_name fieldmay require an alphabetic string as input, and may have a maximum lengthlimit; the patient_age field may require a string of three numerals, andthe leading numeral may have to be “0” or “1”; the patient_gender fieldmay only allow one of two inputs, “Male” and “Female”; apatient_birth_date field may require input in a “MM/DD/YYYY” format;etc.

Typical EHRs are also structured in terms of the vocabulary they use, asclinical terms are normalized to a standard set of terms utilized by theinstitution maintaining the EHR. The standard set of terms may bespecific to the institution, or may be a more widely used standard. Forexample, a clinician dictating or writing a free-form note may use anyof a number of different terms for the condition of a patient currentlysuffering from an interruption of blood supply to the heart, including“heart attack”, “acute myocardial infarction”, “acute MI” and “AMI”. Tofacilitate interoperability of EHR data between various departments andusers in the institution, and/or to allow identical conditions to beidentified as such across patient records for data analysis, a typicalEHR may use only one standardized term to represent each individualclinical concept. For example, “acute myocardial infarction” may be thestandard term stored in the EHR for every case of a heart attackoccurring at the time of a clinical encounter. Some EHRs may representclinical terms in a data format corresponding to a coding standard, suchas the International Classification of Disease (ICD) standard. Forexample, “acute myocardial infarction” may be represented in an EHR as“ICD-9 410”, where 410 is the code number for “acute myocardialinfarction” according to the ninth edition of the ICD standard.

To allow clinicians and other healthcare personnel to enter clinicaldocumentation data directly into an EHR in its discrete structured dataformat, many EHRs are accessed through user interfaces that makeextensive use of point-and-click input methods. While some data items,such as the patient's name, may require input in (structured) textual ornumeric form, many data items can be input simply through the use of amouse or other pointing input device (e.g., a touch screen) to makeselections from pre-set options in drop-down menus and/or sets ofcheckboxes and/or radio buttons or the like.

The inventors have recognized, however, that while some clinicians mayappreciate the ability to directly enter structured data into an EHRthrough a point-and-click interface, many clinicians may prefer beingunconstrained in what they can say and in what terms they can use in afree-form note, and many may be reluctant to take the time to learnwhere all the boxes and buttons are and what they all mean in an EHRuser interface. In addition, many clinicians may prefer to takeadvantage of the time savings that can be gained by providing notesthrough verbal dictation, as speech can often be a faster form of datacommunication than typing or clicking through forms.

Accordingly, some embodiments described herein relate to techniques forenhancing the creation and use of structured electronic medical records,using techniques that enable a clinician to provide input andobservations via a free-form narrative clinician's note. Someembodiments involve the automatic extraction of discrete clinical facts,such as could be stored as discrete structured data items in anelectronic medical record, from a clinician's free-form narration of apatient encounter. In this manner, free-form input may be provided, butthe advantages of storage, maintenance and accessing of clinicaldocumentation data in electronic forms may be maintained. For example,the storage of a patient's clinical documentation data as a collectionof discrete structured data items may provide the benefits of being ableto query for individual data items of interest, and being able toassemble arbitrary subsets of the patient's data items into new reports,orders, invoices, etc., in an automated and efficient manner.

Automatic extraction of clinical facts from a free-form narration may beperformed in any suitable way using any suitable technique(s), asaspects of the present invention are not limited in this respect. Insome embodiments, pre-processing may be performed on a free-formnarration prior to performing fact extraction, to determine the sequenceof words represented by the free-form narration. Such pre-processing mayalso be performed in any suitable way using any suitable technique(s),as aspects of the present invention are not limited in this respect. Forexample, in some embodiments, the clinician may provide the free-formnarration directly in textual form (e.g., using a keyboard or other textentry device), and the textual free-form narration may be automaticallyparsed to determine its sequence of words. In other embodiments, theclinician may provide the free-form narration in audio form as a spokendictation, and an audio recording of the clinician's spoken dictationmay be received and/or stored. The audio input may be processed in anysuitable way prior to or in the process of performing fact extraction,as aspects of the invention are not limited in this respect. In someembodiments, the audio input may be processed to form a textualrepresentation, and fact extraction may be performed on the textualrepresentation. Such processing to produce a textual representation maybe performed in any suitable way. For example, in some embodiments, theaudio recording may be transcribed by a human transcriptionist, while inother embodiments, automatic speech recognition (ASR) may be performedon the audio recording to obtain a textual representation of thefree-form narration provided via the clinician's dictation. Any suitableautomatic speech recognition technique may be used, as aspects of thepresent invention are not limited in this respect. In other embodiments,speech-to-text conversion of the clinician's audio dictation may not berequired, as a technique that does not involve processing the audio toproduce a textual representation may be used to determine what wasspoken. In one example, the sequence of words that was spoken may bedetermined directly from the audio recording, e.g., by comparing theaudio recording to stored waveform templates to determine the sequenceof words. In other examples, the clinician's speech may not berecognized as words, but may be recognized in another form such as asequence or collection of abstract concepts. It should be appreciatedthat the words and/or concepts represented in the clinician's free-formnarration may be represented and/or stored as data in any suitable form,including forms other than a textual representation, as aspects of thepresent invention are not limited in this respect.

In some embodiments, one or more clinical facts may be automaticallyextracted from the free-form narration (in audio or textual form) orfrom a pre-processed data representation of the free-form narrationusing a clinical language understanding (CLU) engine. Such a CLU enginemay be implemented in any suitable form, as aspects of the presentinvention are not limited in this respect. In some embodiments, a CLUengine may be implemented using techniques such as those described inU.S. Pat. No. 7,493,253, entitled “Conceptual World RepresentationNatural Language Understanding System and Method”, which is incorporatedherein by reference in its entirety. Such a CLU engine may make use of aformal ontology linked to a lexicon of clinical terms. The formalontology may be implemented as a relational database, or in any othersuitable form, and may represent semantic concepts relevant to theclinical domain, as well as linguistic concepts related to ways thesemantic concepts may be expressed in natural language.

In some embodiments, concepts in a formal ontology used by a CLU enginemay be linked to a lexicon of clinical terms and/or codes, such thateach clinical term and each code is linked to at least one concept inthe formal ontology. In some embodiments, the lexicon may include thestandard clinical terms and/or codes used by the institution in whichthe CLU engine is applied. For example, the standard clinical termsand/or codes used by an EHR maintained by the institution may beincluded in the lexicon linked to the CLU engine's formal ontology. Insome embodiments, the lexicon may also include additional clinical termsused by the various clinicians within the institution when providingtheir free-form narrations. Such additional clinical terms may belinked, along with their corresponding standard clinical terms, to theappropriate shared concepts within the formal ontology. For example, thestandard term “acute myocardial infarction” as well as othercorresponding terms such as “heart attack”, “acute MI” and “AMI” may allbe linked to the same abstract concept in the formal ontology—a conceptrepresenting an interruption of blood supply to the heart. Such linkageof multiple clinical terms to the same abstract concept in someembodiments may relieve the clinician of the burden of ensuring thatonly standard clinical terms preferred by the institution appear in thefree-form narration. For example, in some embodiments, a clinician maybe free to use the abbreviation “AMI” or the colloquial “heart attack”in his free-form narration, and the shared concept linkage may allow theCLU engine to nevertheless automatically extract a fact corresponding to“acute myocardial infarction”.

In some embodiments, a formal ontology used by a CLU engine may alsorepresent various types of relationships between the conceptsrepresented. One type of relationship between two concepts may be aparent-child relationship, in which the child concept is a more specificversion of the parent concept. In some embodiments, any other type(s) ofrelationship useful to the process of clinical documentation may also berepresented in the formal ontology. For example, one type ofrelationship may be a symptom relationship. In one example of a symptomrelationship, a concept linked to the term “chest pain” may have arelationship of “is-symptom-of” to the concept linked to the term “heartattack”. Other types of relationships may include complicationrelationships, comorbidity relationships, interaction relationships(e.g., among medications), and many others. Any number and type(s) ofconcept relationships may be included in such a formal ontology, asaspects of the present invention are not limited in this respect.

In some embodiments, automatic extraction of clinical facts from aclinician's free-form narration may involve parsing the free-formnarration to identify clinical terms that are represented in the lexiconof the CLU engine. Concepts in the formal ontology linked to theclinical terms that appear in the free-form narration may then beidentified, and concept relationships in the formal ontology may betraced to identify further relevant concepts. Through theserelationships, as well as the linguistic knowledge represented in theformal ontology, one or more clinical facts may be extracted. Forexample, if the free-form narration includes the clinical term“hypertension” and the linguistic context relates to the patient's past,the CLU engine may automatically extract a fact indicating that thepatient has a history of hypertension. On the other hand, if thefree-form narration includes the clinical term “hypertension” in asentence about the patient's mother, the CLU engine may automaticallyextract a fact indicating that the patient has a family history ofhypertension. In some embodiments, relationships between concepts in theformal ontology may also allow the CLU engine to automatically extractfacts containing clinical terms that were not explicitly included in thefree-form narration. For example, the clinical term “meningitis” canalso be described as inflammation in the brain. If the free-formnarration includes the terms “inflammation” and “brain” in proximity toeach other, then relationships in the formal ontology between conceptslinked to the terms “inflammation”, “brain” and “meningitis” may allowthe CLU engine to automatically extract a fact corresponding to“meningitis”, despite the fact that the term “meningitis” was not statedin the free-form narration.

It should be appreciated that the foregoing descriptions are provided byway of example only, and that any suitable technique(s) for extracting aset of one or more clinical facts from a free-form narration may beused, as aspects of the present invention are not limited to anyparticular fact extraction technique.

The inventors have recognized and appreciated that the automaticextraction of clinical facts directly from a free-form narration of apatient encounter provided by a clinician may create the opportunity fornumerous enhancements to processes involved in clinical documentation inhealthcare institutions. Some such enhancements may help make itpossible for a clinician to efficiently oversee a process involvingderiving any one or combination of updated patient records, billinginformation, ordering information, quality of care assurances, decisionsupport, etc., directly from a free-form narration in a singleinteractive session with a clinical fact review system.

In some embodiments, automatic extraction of clinical facts from atextual representation of a clinician's free-form narration (e.g., froma text narrative) of a patient encounter may be enhanced byre-formatting the text narrative to facilitate the automatic extractionof the clinical facts. For example, in some embodiments a CLU enginethat performs the automatic fact extraction may make use of linguisticknowledge that has some dependency on accurate placement of sentenceboundaries in the text narrative. Accordingly, in some embodiments, thefact extraction may be enhanced by adding, removing and/or correctingsentence boundaries in the text narrative to comply with the linguisticstructure expected by the CLU engine. Examples of ways in which sentenceboundary pre-processing can be implemented are described below. Inanother example, automatic fact extraction may be enhanced bynormalizing section headings in the text narrative to comply withstandard section headings used by the healthcare institution for whichthe clinical documentation is being performed.

In some embodiments, a linkage may be maintained between each extractedclinical fact and the portion of the free-form narration from which thatfact was extracted. For example, if a fact corresponding to “acutemyocardial infarction” is extracted from a free-form narration becauseit included the term “heart attack”, a linkage may be maintained betweenthat extracted fact and the words “heart attack” in the free-formnarration. In some embodiments, while the clinician or another user isreviewing the extracted clinical facts via a user interface to a factreview system, the system may provide one or more indicators to the user(who may be the clinician himself or a different person) of thedifferent linkages between the different extracted facts and theportions of the free-form narration from which they were extracted. Suchindicators may be visual indicators, audio indicators, or any othersuitable type of indicators, as aspects of the present invention are notlimited in this respect. In some embodiments, such linkage indicatorsmay enhance the ability of the clinician or other user to review theextracted facts for accuracy, with reference to the specific parts ofthe free-form narration that generated them. In some embodiments, if atextual representation of the free-form narration has been re-formattedprior to fact extraction, linkages may still be maintained between theextracted facts and the original text narrative, to allow the user torelate the extracted facts to the narration as it was originally givenby the clinician. While some embodiments provide linkage information foreach extracted fact, it should be appreciated that aspects of theinvention relating to providing linkage information are not so limited,as linkage information may be provided for one or any subset of theextracted facts.

In some embodiments, automatically extracted clinical facts may also beautomatically reviewed, and automatic alerts may be provided to theclinician or other user if opportunities are identified for the clinicaldocumentation of the patient encounter to be improved. Such alerts maybe visual alerts, audio alerts, or any other suitable type of alerts, asaspects of the present invention are not limited in this respect. Insome embodiments, such alerts may be provided to the clinician or otheruser at a time subsequent to the completion of the patient encounter,and may provide the opportunity for the clinician or other user toprovide additional information that was ascertained from the patientencounter but was not originally specified in the free-form narration.In other embodiments, such alerts may be provided to the clinician whilethe patient encounter is still in progress, and may provide theopportunity for the clinician to initiate further interaction with thepatient to ascertain additional information to include in the clinicaldocumentation.

In some embodiments, a fact review system may be programmed with a setof deterministic rules to trigger alerts. For example, a set ofdeterministic rules may specify that certain extracted facts, certaincombinations of extracted facts, certain combinations of extracted factsand terms in the free-form narration, and/or certain combinations offacts extracted from the current patient encounter and facts from thepatient's previous history automatically trigger alerts to the user. Inother embodiments, the fact review system may be programmed to undertakea probabilistic analysis or apply a statistical model to determinewhether information specified in the free-form narration will triggeralerts to the user. It should be appreciated, however, that a factreview system in accordance with embodiments described herein is notlimited to any particular programming technique, as any suitable suchtechnique may be used. In addition, it should be appreciated thatautomatic alerts may also be provided in embodiments that do not involveautomatic extraction of clinical facts from a free-form narration. Forexample, such alerts may also be triggered by clinical facts received asdiscrete structured data items, such as direct input to an electronicmedical record such as an EHR. It should thus be appreciated that alertsmay be provided based on analysis of clinical facts collected in anysuitable way, as aspects of the present invention are not limited inthis respect.

In some embodiments, an alert may be provided when a set of one or moreclinical facts is collected from a patient encounter, and it isdetermined that there is an opportunity to increase the specificity ofthe set of facts. In some embodiments, it may be determined that anadditional fact may possibly be ascertained from the patient encounter,and that the additional fact would add specificity to the set ofclinical facts already collected from the patient encounter. In oneexample, such an additional fact may be a more specific version of oneof the original facts, and the specificity of the set of facts may beincreased by replacing the original fact with its more specific version,provided that it can truly be ascertained from the patient encounter.For instance, the original fact may describe a condition, and the morespecific version may describe the same condition as “acute” or“chronic”. In another example, two or more of the original facts, whenappearing in combination, may imply an additional fact, and documentingthe additional fact may increase the specificity of the record of thepatient encounter. In some embodiments, an alert may query the user asto whether an additional fact should actually be ascertained from thepatient encounter, and may allow the user to increase the specificity ofthe facts by documenting the additional fact.

In some embodiments, an alert may be provided when a set of one or moreclinical facts is collected from a patient encounter, and it isdetermined that a diagnosis that was not specified in the set of factsmay possibly be ascertained from the patient encounter. In one example,such an unspecified diagnosis may be a known comorbidity of a diagnosisthat was included in the set of facts. In another example, theunspecified diagnosis may be a known complication of a procedure ordiagnosis included in the set of facts. In yet another example, theunspecified diagnosis may be an identification of the fact that adiagnosis included in the set of facts is actually a complication of aprocedure or other diagnosis included in the set of facts, or of aprocedure or other diagnosis included in facts from the patient'shistory prior to the current encounter. Similarly, the unspecifieddiagnosis may be an identification of the fact that a diagnosis includedin facts from the patient's previous history is a complication of adiagnosis ascertained during the current patient encounter. In someembodiments, when the possibility or likelihood of such an unspecifieddiagnosis is determined from the original set of facts collected fromthe patient encounter, an alert may query the user (e.g., the clinicianor another user) as to whether the unspecified diagnosis should beascertained from the patient encounter.

In some embodiments, an alert may be provided when a set of one or moreclinical facts is collected from a patient encounter, and it isdetermined that two or more of the facts in the set conflict with eachother in some way, or it is determined that one or more of the facts inthe set conflict with one or more facts in the patient's history. Insome embodiments, a fact review system may be programmed toautomatically generate such alerts based on a known set of combinationsof facts that have undesirable interactions. For example, an alert maybe generated when the set of facts indicate that the patient has beenprescribed a certain medication (drug A) in addition to a certain othermedication (drug B) with which it negatively interacts, such that thetwo medications should not be prescribed together. In some embodiments,the prescriptions of both drug A and drug B may be specified in the setof facts collected from the current patient encounter, while in otherembodiments, the prescription of drug A may be specified in a fact fromthe current patient encounter, and the prescription of drug B may bespecified in a fact contained in a record of the patient's history withthe institution. Thus, in some embodiments, the fact review system mayaccess both facts collected from a current patient encounter and factsfrom the patient's historical records to determine whether alerts shouldbe generated. In some embodiments, an alert to a conflict may betriggered by a combination of facts, at least one of which does notcorrespond to a medication. For example, alerts may be provided forcontraindications related to a combination of a medication with anallergy, a medication with a diagnosis, a medication with a patient'sage or gender, a medication with a condition indicated in the patient'shistory, a medical procedure with any of the foregoing characteristics,or any other combination of a planned treatment with another clinicalfact from the current patient encounter or from the patient's historyfor which the planned treatment is known to be contraindicated.

In some embodiments, an alert may be provided when a set of one or moreclinical facts is collected from a patient encounter, and it isdetermined that there is an opportunity to add to the clinicaldocumentation of the patient encounter for quality review purposes. Insome embodiments, a fact review system may be programmed with a set ofdeterministic rules to generate automatic alerts in response to certainfacts or certain combinations of facts, based on a standard set ofquality of care measures. Such a quality of care standard may beproprietary and unique to the specific healthcare institution or may bea standard that is not institution specific, such as that of thePhysician Quality Reporting Initiative (PQRI) or that of the JointCommission on Accreditation of Healthcare Organizations (JCAHO). Anysuitable quality of care standard may be used, as aspects of the presentinvention are not limited to any particular quality of care standard. Insome embodiments, when a collected fact or combination of facts isassociated with a certain recommended action on the part of theclinician according to the quality of care standard, an alert may beprovided to query the user as to whether the recommended action wasperformed.

In some embodiments, a mechanism may be provided to adaptively filterthe automatic alerts generated by the fact review system, by learningfrom the clinician's or other user's interaction with the system overtime. For example, if it is determined that a particular userconsistently ignores a particular type of alert, the system may stopissuing similar alerts when they are triggered by future facts. In someembodiments, the adaptive learning may be specific to each individualuser and may help to prevent alert fatigue, which may involvefrustration at repeatedly being bothered by an alert that the user doesnot find relevant. In some embodiments, the adaptive learning mayinvolve the collection of data regarding patterns of facts that tend tobe present when the user ignores alerts, and the system may filter outfuture alerts that match those patterns of facts. In some embodiments,adaptive alert filtering may be performed based on rules or statisticalusage patterns on an institutional level, such that alerts notconsidered relevant for the specific healthcare institution in which thefact review system is operating are not provided.

In some embodiments, a human user other than the clinician may reviewthe set of clinical facts collected from a patient encounter, and maymanually (e.g., not automatically, but involving human action) cause oneor more alerts to be issued to the clinician that were not issuedautomatically by the fact review system. Such a human user may manuallycause alerts to be issued in any suitable way, as aspects of theinvention are not limited in this respect. In one example, the humanuser may provide instructional input to the fact review system to causethe fact review system to generate an alert specified by the human user.In other examples, the human user may use a different method and/orsystem, other than the fact review system, to issue an alert to theclinician. Such a different method in some embodiments need not bemachine-based, as aspects of the invention are not limited in thisrespect. In some embodiments, the human user may have access to thepatient's past medical history within and/or external to the healthcareinstitution, for example in the form of an electronic medical recordand/or past clinical documents relating to the patient's care at theinstitution and/or elsewhere. In some embodiments, the human user maymake reference to this past medical history, in addition to the clinicalfacts from the current patient encounter, to determine whether tomanually cause an alert to be issued to the clinician. In someembodiments, the human user may determine to issue an alert, similar toany of the various types of automatic alerts described above, if thefacts and the patient's history indicate a situation in which theautomatic fact review system should have generated an automatic alert,but it failed to accurately recognized the situation. In someembodiments, if the clinician chose to ignore an alert automaticallygenerated by the fact review system, but ignoring such an alert wascontrary to the policy of the institution, the human reviewer maydetermine to manually issue a follow-up alert to the clinician. Thus, insome embodiments, an automatic fact review system may coexist in aninstitutional setting with a manual review process involving a humanuser, and the manual review process may provide back-up and/oradditional functionality to complement the automatic fact reviewprocesses.

In some embodiments, when clinical facts are extracted from a free-formnarration, a CLU engine may encounter situations in which disambiguationis desired between multiple facts that could potentially be extractedfrom the same portion of the free-form narration. For example, a term inthe free-form narration might be linked to two different concepts in theformal ontology used by the CLU engine, and it might not be likely thatboth of those concepts were intended to coexist in the free-formnarration. In such situations, a fact review system in some embodimentsmay provide a structured choice to the user to disambiguate betweenmultiple facts tentatively extracted by the CLU engine. In someembodiments, each of the options provided in the structured choice maycorrespond to one of the multiple tentative facts, and the user maychoose one of the options to specify which fact should actually beextracted from the free-form narration.

In some embodiments, when the user makes a selection of a fact presentedthrough a structured choice provided by the fact review system, atextual representation of the clinician's free-form narration mayautomatically be updated to explicitly identify the selected fact ashaving been ascertained from the patient encounter. For example, if thefree-form narration originally included a term linked to two differentconcepts in the CLU engine's ontology, the fact review system couldpresent the user a structured choice between a different term linkedonly to one of the concepts and a different term linked only to theother of the concepts. When the user selects one of the different termsin the structured choice presented, in some embodiments the textnarrative may automatically be updated to replace the original term withthe selected term. In some embodiments, such updating of the textnarrative may be performed in response to any type of user selection ofan option provided by the fact review system, corresponding to aclinical fact that could possibly be ascertained from the patientencounter. Some examples include disambiguating options, optionscorresponding to additional facts for increased specificity and optionscorresponding to unspecified diagnoses, as discussed above. In someembodiments, rather than replacing text in the narrative, new textcorresponding to the selected fact may be generated and simply added tothe narrative in one or more appropriate locations. In some embodiments,the location(s) at which to insert text identifying the selected factmay be automatically determined by identifying one or more sectionheadings in the text narrative, and by inserting the text in the sectionor sections most closely corresponding to the selected fact.

In some embodiments, a fact review system may allow a clinician or otheruser to directly add a clinical fact as a discrete structured data item,and to indicate a linkage to a portion of the clinician's free-formnarration of the patient encounter from which the added fact should havebeen extracted. For example, the user may specify a clinical fact as adiscrete structured data element, select a word or set of words (whichneed not be contiguous) in the free-form narration, and indicate thatthe specified fact is ascertained from that portion (e.g., that word orset of words) of the free-form narration. In some embodiments, when sucha fact is added, the CLU engine may be updated for that user (or for theclinician who provided the free-form narration) to link the selectedword(s) from the free-form narration to one or more concepts in theformal ontology corresponding to the added fact. In some embodiments,the free-form narration may further be re-processed by the updated CLUengine to extract any further additional facts that may be determinedbased on the updated terminology. In one example, if the user selected aword in the patient history section of the free-form narration, andadded a fact specifying that the patient has a history of a particularcondition, the updated CLU engine re-processing the free-form narrationmight identify the same word in the family history section, and extractan additional fact that the patient has a family history of the samecondition. In some embodiments, such automatic re-processing may sparethe clinician or other user the time and effort that otherwise would berequired to define multiple facts corresponding to the same terminologyin the free-form dictation. In some embodiments, similar re-processingmay be performed when the user edits or deletes a fact originallyextracted automatically from the free-form narration, when the fact islinked to terminology that appears in multiple parts of the free-formnarration.

In some embodiments, as discussed above, a fact review system may allowa user to add, delete and/or modify (collectively referred to as“change”) a clinical fact extracted from a free-form narration of apatient encounter provided by a clinician, resulting in a change to theset of extracted facts. In some instances, one or more such changes madeto the set of facts corresponding to the current patient encounter maycreate one or more inconsistencies between the set of facts and thesemantic content of the original free-form narration. For example, aclinician may originally specify a particular diagnosis in a free-formnarration, and a CLU engine may extract a clinical fact corresponding tothat diagnosis. If the clinician later changes his mind and would liketo replace the original diagnosis with a different diagnosis, he mayhave the option in some embodiments of simply editing the extracted factdirectly, rather than editing the data representation of the free-formnarration itself. Such a situation may create an inconsistency betweenthe free-form narration and the corresponding set of clinical facts, asthe facts may now specify the new diagnosis, and the free-form narrationmay still specify the original diagnosis. In such situations, the factreview system in some embodiments may alert the clinician or other userto the inconsistency, and/or may provide any of several options to theuser to address the inconsistency. One option may be to ignore theinconsistency and allow it to persist in the clinical documentation.Another option may be to allow the user to edit the data representationof the free-form narration to be consistent with the current set ofclinical facts. Another option may be to allow the system toautomatically update the data representation of the free-form narrationby adding, deleting or replacing one or more portions of the free-formnarration. Yet another option may be simply to append a note to thefree-form narration, indicating and optionally explaining theinconsistency.

In some embodiments, as discussed above, a clinical fact review systemmay provide various tools for a clinician to review and/or edit factscorresponding to a current patient encounter, receive alerts generatedbased on those facts, review and/or edit a free-form narration of thepatient encounter provided by the clinician, and/or review the linkagesmaintained between clinical facts extracted by a CLU engine and theportions of the free-form narration from which the clinical facts wereextracted. Such tools may be provided in any suitable form, includingvisual forms, audio forms, combined forms or any other form providingthe functionality described herein, as aspects of the present inventionare not limited in this respect. When the tools are provided in visualform, their functionality may be accessed through a graphical userinterface (GUI). In some embodiments, the GUI may be organized in a wayto allow the human user(s) to efficiently process the informationdisplayed. For example, in some embodiments, text narratives, facts andalerts may be displayed in consistent locations within the userinterface and organized by type and/or priority. Different colors,textual styles and/or graphical styles may be utilized to direct theuser's attention to high-priority alerts, and/or to make linkagesbetween related items in the display easily recognizable. In someembodiments, the organization and/or composition of such a visualdisplay may be determined in accordance with principles used in thedevelopment of heads-up displays (HUDs).

In some embodiments, a fact review system operating on a set of clinicalfacts ascertained from a patient encounter may provide tools forpromoting efficiency in the workflow of the clinician and/or otherpersonnel beyond the conclusion of the patient encounter. For example,in some embodiments, the fact review system may interface with one ormore Computerized Physician Order Entry (CPOE) systems to automaticallyplace orders for prescriptions, laboratory tests, radiology screenings,surgical or other medical procedures and/or other planned treatmentaction items, based on such items (e.g., medication names, dosages,procedure names, dates, etc.) being specified in the set of factscorresponding to the current patient encounter. In some embodiments,such items may be identified based on their being extracted from a“plan” section of a free-form narration. In some embodiments, the factreview system may interface with one or more scheduling systems toschedule appointments for medical procedures and/or future office visitswithin or external to the institution. In some embodiments, the factreview system may format one or more facts into a standard orproprietary messaging format to facilitate interfacing with any of suchsystems. In some embodiments, billing reports, patient dischargeinstructions and/or other documents may be automatically generated orinitially populated based on the set of clinical facts. In someembodiments with any of the above-described functionality, the factreview system may provide an alert to the user and/or may prompt foruser or clinician approval prior to taking any of the above actions.

In some embodiments, a fact review system may provide tools forevidence-based clinical decision support based on the set of clinicalfacts collected for the current patient encounter. In some embodiments,the fact review system may have access to one or more data sets of pastpatient reports and/or one or more archives of medical literaturedocuments that may provide information regarding various conditions,treatment outcomes and the like that are relevant to the current patientencounter. In some embodiments, the available documents may have beenprocessed by the CLU engine and indexed using the same system ofterminology used to extract clinical facts from free-form clinicalnarrations. As such, in some embodiments, the facts corresponding to thecurrent patient encounter may be efficiently matched to relevantavailable documents, and those documents or a subset thereof may beretrieved for display or otherwise provided to the clinician to aid inhis determination of a treatment plan for the current patient. In someembodiments, a statistical model may be trained on the data set of pastpatient outcomes and/or on data in the medical literature, such that thesystem may go beyond mere presentation of references to actually predictbest courses of treatment by applying the statistical model to thecollection of facts corresponding to the current patient encounterand/or to the patient's medical history. In some embodiments, treatmentrecommendations may be provided to the clinician along with links toreferences in the literature or other available data supporting therecommendations. In some embodiments, CLU indexing of large quantitiesof patient records and/or literature documents may also be used tofacilitate clinical research studies, as available natural languagedocuments may be efficiently mapped to an ad hoc query corresponding toa research question. From the resulting corpus of conceptually relevantdocuments, treatment outcomes and/or other required information or factsmay be extracted using CLU technology to aid in synthesizing an answerto the research question.

While a number of inventive features for clinical documentationprocesses are described above, it should be appreciated that embodimentsof the present invention may include any one of these features, anycombination of two or more features, or all of the features, as aspectsof the invention are not limited to any particular number or combinationof the above-described features. The aspects of the present inventiondescribed herein can be implemented in any of numerous ways, and are notlimited to any particular implementation techniques. Described below areexamples of specific implementation techniques; however, it should beappreciate that these examples are provided merely for purposes ofillustration, and that other implementations are possible.

One illustrative application for the techniques described herein is foruse in a system for enhancing clinical documentation processes. Anexemplary operating environment for such a system is illustrated inFIG. 1. The exemplary operating environment includes a server 100communicatively connected via any suitable communication medium or media(e.g., local and/or network connections) to terminals 110 and 140.Server 100 and terminals 110 and 140 each may be implemented in anysuitable form, as aspects of the present invention are not limited inthis respect. For example, each may be implemented as a singlestand-alone machine, or may be implemented by multiple distributedmachines that share processing tasks in any suitable manner. Any or allof the machines designated as server 100 and terminals 110 and 140 maybe implemented as one or more computers; an example of a suitablecomputer is described below. In some embodiments, each of server 100 andterminals 110 and 140 may include one or more non-transitorycomputer-readable storage devices storing processor-executableinstructions, and one or more processors that execute theprocessor-executable instructions to perform the functions describedherein. The storage devices may be implemented as computer-readablestorage media encoded with the processor-executable instructions;examples of suitable computer-readable storage media are discussedbelow.

As depicted, server 100 includes an ASR engine 102, a CLU engine 104,and a fact review component 106. Each of these processing components ofserver 100 may be implemented in software, hardware, or a combination ofsoftware and hardware. Components implemented in software may comprisesets of processor-executable instructions that may be executed by theone or more processors of server 100 to perform the functionalitydescribed herein. Each of ASR engine 102, CLU engine 104 and fact reviewcomponent 106 may be implemented as a separate component of server 100,or any combination of these components may be integrated into a singlecomponent or a set of distributed components. In addition, any one ofASR engine 102, CLU engine 104 and fact review component 106 may beimplemented as a set of multiple software and/or hardware components. Itshould be understood that any such component depicted in FIG. 1 is notlimited to any particular software and/or hardware implementation and/orconfiguration.

As illustrated in FIG. 1, terminal 110 is operated by a clinician 120,who may be a physician, a physician's aide, a nurse, or any otherpersonnel involved in the evaluation and/or treatment of a patient 122in a clinical setting. During the course of a clinical encounter withpatient 122, or at some point thereafter, clinician 120 may wish todocument the patient encounter. Such a patient encounter may include anyinteraction between clinician 120 and patient 122 in a clinicalevaluation and/or treatment setting, including, but not limited to, anoffice visit, an interaction during hospital rounds, an outpatient orinpatient procedure (surgical or non-surgical), a follow-up evaluation,a visit for laboratory or radiology testing, etc. One method thatclinician 120 may use to document the patient encounter may be to enterclinical facts that can be ascertained from the patient encounter intoterminal 110 as discrete structured data items. The set of clinicalfacts, once entered, may be transmitted in some embodiments via anysuitable communication medium or media (e.g., local and/or networkconnection(s) that may include wired and/or wireless connection(s)) toserver 100. Specifically, in some embodiments, the set of clinical factsmay be received at server 100 by a fact review component 106, exemplaryfunctions of which are described below.

Another method that may be used by clinician 120 to document the patientencounter is to provide a free-form narration of the patient encounter.In some embodiments, the narration may be free-form in the sense thatclinician 120 may be unconstrained with regard to the structure andcontent of the narration, and may be free to provide any sequence ofwords, sentences, paragraphs, sections, etc., that he would like. Insome embodiments, there may be no limitation on the length of thefree-form narration, or the length may be limited only by the processingcapabilities of the user interface into which it is entered or of thelater processing components that will operate upon it. In otherembodiments, the free-form narration may be constrained in length (e.g.,limited to a particular number of characters).

A free-form narration of the patient encounter may be provided byclinician 120 in any of various ways. One way may be to manually enterthe free-form narration in textual form into terminal 110, e.g., using akeyboard. In this respect, the one or more processors of terminal 110may in some embodiments be programmed to present a user interfaceincluding a text editor/word processor to clinician 120. Such a texteditor/word processor may be implemented in any suitable way, as aspectsof the present invention are not limited in this respect.

Another way to provide a free-form narration of the patient encountermay be to verbally speak a dictation of the patient encounter. Such aspoken dictation may be provided in any suitable way, as aspects of thepresent invention are not limited in this respect. As illustrated inFIG. 1, one way that clinician 120 may provide a spoken dictation of thefree-form narration may be to speak the dictation into a microphone 112operatively connected (e.g., via a direct wired connection, a directwireless connection, or via a connection through an intermediate device)to terminal 110. An audio recording of the spoken dictation may then bestored in any suitable data format, and transmitted to server 100.Another way that clinician 120 may provide the spoken dictation may beto speak into a telephone 118, from which an audio signal may betransmitted to be recorded at server 100, at the site of medicaltranscriptionist 130, or at any other suitable location. Alternatively,the audio signal may be recorded in any suitable data format at anintermediate facility, and the audio data may then be relayed to server100 or to medical transcriptionist 130.

In some embodiments, medical transcriptionist 130 may receive the audiorecording of the dictation provided by clinician 120, and may transcribeit into a textual representation of the free-form narration (e.g., intoa text narrative). Medical transcriptionist 130 may be any human wholistens to the audio dictation and writes or types what was spoken intoa text document. In some embodiments, medical transcriptionist 130 maybe specifically trained in the field of medical transcription, and maybe well-versed in medical terminology. In some embodiments, medicaltranscriptionist 130 may transcribe exactly what she hears in the audiodictation, while in other embodiments, medical transcriptionist 130 mayadd formatting to the text transcription to comply with generallyaccepted medical document standards. When medical transcriptionist 130has completed the transcription of the free-form narration into atextual representation, the resulting text narrative may in someembodiments be transmitted to server 100 or any other suitable location(e.g., to a storage location accessible to server 100). Specifically, insome embodiments the text narrative may be received from medicaltranscriptionist 130 by CLU engine 104 within server 100. Exemplaryfunctionality of CLU engine 104 is described below.

In some other embodiments, the audio recording of the spoken dictationmay be received, at server 100 or any other suitable location, byautomatic speech recognition (ASR) engine 102. In some embodiments, ASRengine 102 may then process the audio recording to determine what wasspoken. As discussed above, such processing may involve any suitablespeech recognition technique, as aspects of the present invention arenot limited in this respect. In some embodiments, the audio recordingmay be automatically converted to a textual representation, while inother embodiments, words identified directly from the audio recordingmay be represented in a data format other than text, or abstractconcepts may be identified instead of words. Examples of furtherprocessing are described below with reference to a text narrative thatis a textual representation of the free-form narration; however, itshould be appreciated that similar processing may be performed on otherrepresentations of the free-form narration as discussed above. When atextual representation is produced, in some embodiments it may bereviewed by a human (e.g., a transcriptionist) for accuracy, while inother embodiments the output of ASR engine 102 may be accepted asaccurate without human review.

In some embodiments, ASR engine 102 may make use of a lexicon ofclinical terms (which may be part of, or in addition to, another moregeneral speech recognition lexicon) while determining the sequence ofwords that were spoken in the free-form narration provided by clinician120. However, aspects of the invention are not limited to the use of alexicon, or any particular type of lexicon, for ASR. When used, theclinical lexicon in some embodiments may be linked to a clinicallanguage understanding ontology utilized by CLU engine 104, such thatASR engine 102 might produce a text narrative containing terms in a formunderstandable to CLU engine 104. In some embodiments, a more generalspeech recognition lexicon might also be shared between ASR engine 102and CLU engine 104. However, in other embodiments, ASR engine 102 maynot have any lexicon intentionally in common with CLU engine 104. Insome embodiments, a lexicon used by ASR engine 102 may be linked to adifferent type of clinical ontology, such as one not designed or usedfor language understanding. It should be appreciated that any lexiconused by ASR engine 102 and/or CLU engine 104 may be implemented and/orrepresented as data in any suitable way, as aspects of the invention arenot limited in this respect.

In some embodiments, a text narrative, whether produced by ASR engine102 (and optionally verified or not by a human), produced by medicaltranscriptionist 130, directly entered in textual form through terminal110, or produced in any other way, may be re-formatted in one or moreways before being received by CLU engine 104. Such re-formatting may beperformed by ASR engine 102, by a component of CLU engine 104, by acombination of ASR engine 102 and CLU engine 104, or by any othersuitable software and/or hardware component. In some embodiments, there-formatting may be performed in a way known to facilitate factextraction, and may be performed for the purpose of facilitating theextraction of clinical facts from the text narrative by CLU engine 104.For example, in some embodiments, processing to perform fact extractionmay be improved if sentence boundaries in the text narrative areaccurate. Accordingly, in some embodiments, the text narrative may bere-formatted prior to fact extraction to add, remove or correct one ormore sentence boundaries within the text narrative. In some embodiments,this may involve altering the punctuation in at least one locationwithin the text narrative. In another example, fact extraction may beimproved if the text narrative is organized into sections with headings,and thus the re-formatting may include determining one or more sectionboundaries in the text narrative and adding, removing or correcting oneor more corresponding section headings. In some embodiments, there-formatting may include normalizing one or more section headings(which may have been present in the original text narrative and/or addedor corrected as part of the re-formatting) according to a standard forthe healthcare institution corresponding to the patient encounter (whichmay be an institution-specific standard or a more general standard forsection headings in clinical documents). In some embodiments, a user(such as clinician 120, medical transcriptionist 130, or another user)may be prompted to approve the re-formatted text.

Any suitable technique(s) for implementing re-formatting, examples ofwhich are described above, may be employed, as aspects of the inventionare not limited in this respect. One exemplary technique suitable forperforming re-formatting of a text narrative is described in U.S. patentapplication Ser. No. 11/322,971, filed on Dec. 30, 2005, entitled“Translating Literal Speech to Formatted Text”, which is incorporatedherein by reference in its entirety. Another exemplary technique thatmay be used in some embodiments for performing re-formatting of a textnarrative involves the use of word N-gram statistical models to predictsentence and/or section boundaries in a text narrative. Such statisticalmodels may be trained on a corpus of documents (e.g., past medicalrecords) with correct punctuation and/or section headings (e.g.,supplied by a medical transcriptionist).

In some embodiments, a statistical model may add punctuation (e.g.,periods, exclamation points, question marks, etc.) to add one or moresentence boundaries to a text narrative by computing a probability, foreach word in the text narrative, that a particular punctuation markshould follow that word. In computing the probability that a word shouldbe followed by a punctuation mark, the statistical model may considerthe N-word sequence from the text narrative that ends with that word,and determine the frequency with which that N-word sequence is followedby that punctuation mark in the training data for the statistical model.A lattice may then be constructed using the computed probabilities forall the words in the text narrative, or in a portion of the textnarrative, and the best path in terms of combined probability throughthe lattice may be determined. Where punctuation marks are located inthe best path through the lattice, those punctuation marks may be addedin those locations to the text narrative in producing the formattedtext. In some embodiments, another statistical model may add sectionheadings, corresponding to section boundaries, in a similar fashion. Forexample, in some embodiments, a statistical model for section headingsmay compute probabilities, for each word, that the word should befollowed by a section boundary. In some embodiments, in computingprobabilities, a statistical model for section headings may considermore words that follow the current word than words that precede thecurrent word. In some embodiments, one or more separate statisticalmodels may be trained to delete incorrect sentence and/or sectionboundaries. Those models in some embodiments may be trained throughfeedback from clinician 120 or another user, by observing word sequences(initially including punctuation and/or section boundaries) from whichclinician 120 or another user tends to remove the punctuation and/orsection boundaries when editing.

In some embodiments, either an original or a re-formatted text narrativemay be received by CLU engine 104, which may perform processing toextract one or more clinical facts from the text narrative. The textnarrative may be received from ASR engine 102, from medicaltranscriptionist 130, directly from clinician 120 via terminal 110, orin any other suitable way. Any suitable technique(s) for extractingclinical facts from the text narrative may be used, as aspects of thepresent invention are not limited in this respect. Exemplary techniquesfor clinical fact extraction are described above, and may involve theuse of a clinical language understanding ontology with concepts linkedto a lexicon of clinical terms.

In some embodiments, a user such as clinician 120 may monitor, controland/or otherwise interact with the fact extraction and/or fact reviewprocess through a user interface provided in connection with server 100.One exemplary implementation of such a user interface is graphical userinterface (GUI) 200, illustrated in FIG. 2. In some embodiments, whenthe user is clinician 120, GUI 200 may be presented on a visual display114 of terminal 110, and data displayed via GUI 200 may be downloaded toterminal 110 from server 100. In some embodiments, a user may be personother than a clinician; for example, another person such as codingspecialist 150 may be presented with GUI 200 via visual display 144 ofterminal 140. However, the user interface is not limited to a graphicaluser interface, as other ways of providing data to users from server 100may be used. For example, in some embodiments, audio indicators may betransmitted from server 100 and conveyed to a user (e.g., via speaker116 and/or speaker 146). It should be appreciated that any type of userinterface may be provided in connection with fact extraction, factreview and/or other related processes, as aspects of the invention arenot limited in this respect. While exemplary embodiments as illustratedin FIG. 1 involve data processing at server 100 and data communicationbetween server 100 and terminals 110 and/or 140, it should beappreciated that in other embodiments any or all processing componentsof server 100 may instead be implemented locally at terminal 110 and/orterminal 140, as aspects of the invention are not limited to anyparticular distribution of local and/or remote processing capabilities.

As depicted in FIG. 2, GUI 200 includes a number of separate panesdisplaying different types of data. Identifying information pane 210includes general information identifying patient 222 as a male patientnamed John Doe. Such general patient identifying information may beentered by clinician 120, or by other user 150, or may be automaticallypopulated from an electronic medical record for patient 122, or may beobtained from any other suitable source. Identifying information pane210 also displays the creation date and document type of the reportcurrently being worked on. This information may also be obtained fromany suitable source, such as from stored data or by manual entry. Whenreferring herein to entry of data by clinician 120 and/or other user150, it should be appreciated that any suitable form of data entry maybe used, including input via mouse, keyboard, touchscreen, stylus,voice, or any other suitable input form, as aspects of the invention arenot limited in this respect.

GUI 200 as depicted in FIG. 2 includes a text panel 220 in which a textnarrative referring to the encounter between clinician 120 and patient122 is displayed. In some embodiments, text panel 220 may include texteditor functionality, such that clinician 120 may directly enter thetext narrative into text panel 220, either during the patient encounteror at some time thereafter. If ASR is used to produce the text narrativefrom a spoken dictation provided by clinician 120, in some embodimentsthe text may be displayed in text panel 220 as it is produced by ASRengine 102, either in real time while clinician 120 is dictating, orwith a larger processing delay. In other embodiments, the text narrativemay be received as stored data from another source, such as from medicaltranscriptionist 130, and may be displayed in completed form in textpanel 220. In some embodiments, the text narrative may then be edited ifdesired by clinician 120 and/or other user 150 within text panel 220.However, text editing capability is not required, and in someembodiments text panel 220 may simply display the text narrative withoutproviding the ability to edit it.

Exemplary GUI 200 further includes a fact panel 230 in which one or moreclinical facts, once extracted from the text narrative and/or entered inanother suitable way, may be displayed as discrete structured dataitems. When clinician 120 and/or other user 150 is ready to direct CLUengine 104 to extract one or more clinical facts from the textnarrative, in some embodiments he or she may select process button 240via any suitable selection input method. However, a user indication tobegin fact extraction is not limited to a button such as process button240, as any suitable way to make such an indication may be provided byGUI 200. In some embodiments, no user indication to begin factextraction may be required, and CLU engine 104 may begin a factextraction process as soon as a requisite amount of text (e.g., enoughtext for CLU engine 104 to identify one or more clinical facts that canbe ascertained therefrom) is entered and/or received. In someembodiments, a user may select process button 240 to cause factextraction to be performed before the text narrative is complete. Forexample, clinician 120 may dictate, enter via manual input and/orotherwise provide a part of the text narrative, select process button240 to have one or more facts extracted from that part of the textnarrative, and then continue to provide further part(s) of the textnarrative. In another example, clinician 120 may provide all or part ofthe text narrative, select process button 240 and review the resultingextracted facts, edit the text narrative within text pane 220, and thenselect process button 240 again to review how the extracted facts maychange.

In some embodiments, one or more clinical facts extracted from the textnarrative by CLU engine 104 may be displayed to the user via GUI 200 infact panel 230. Screenshots illustrating an example display of clinicalfacts extracted from an example text narrative are provided in FIGS. 3Aand 3B. FIG. 3A is a screenshot with fact panel 230 scrolled to the topof a display listing clinical facts extracted from the example textnarrative, and FIG. 3B is a screenshot with fact panel 230 scrolled tothe bottom of the display listing the extracted clinical facts. In someembodiments, as depicted in FIGS. 3A and 3B, clinical factscorresponding to a patient encounter may be displayed in fact panel 230,and organized into a number of separate categories of types of facts. Anexemplary set of clinical fact categories includes categories forproblems, medications, allergies, social history, procedures and vitalsigns. However, it should be appreciated that any suitable factcategories may be used, as aspects of the invention are not limited inthis respect. In addition, organization of facts into categories is notrequired, and displays without such organization are possible. Asdepicted in FIGS. 3A and 3B, in some embodiments GUI 200 may beconfigured to provide a navigation panel 300, with a selectableindication of each fact category available in the display of fact panel230. In some embodiments, when the user selects one of the categorieswithin navigation panel 300 (e.g., by clicking on it with a mouse,touchpad, stylus, or other input device), fact panel 230 may be scrolledto display the corresponding fact category. As depicted in FIGS. 3A and3B, all available fact categories for the current document type aredisplayed, even if a particular fact category includes no extracted orotherwise entered clinical facts. However, this is not required; in someembodiments, only those fact categories having facts ascertained fromthe patient encounter may be displayed in fact panel 230.

Fact panel 230 scrolled to the top of the display as depicted in FIG. 3Ashows problem fact category 310, medications fact category 320, andallergies fact category 330. Within problem fact category 310, fourclinical facts have been extracted from the example text narrative; noclinical facts have been extracted in medications fact category 320 orin allergies fact category 330. Within problem fact category 310, fact312 indicates that patient 122 is currently presenting with unspecifiedchest pain; that the chest pain is a currently presenting condition isindicated by the status “active”. Fact 314 indicates that patient 122 iscurrently presenting with shortness of breath. Fact 316 indicates thatthe patient has a history (status “history”) of unspecified essentialhypertension. Fact 318 indicates that the patient has a history ofunspecified obesity. As illustrated in FIG. 3A, each clinical fact inproblem fact category 310 has a name field and a status field. In someembodiments, each field of a clinical fact may be a structured componentof that fact represented as a discrete structured data item. In thisexample, the name field may be structured such that only a standard setof clinical terms for problems may be available to populate that field.For example, the status field may be structured such that only statusesin the Systematized Nomenclature of Medicine (SNOMED) standard (e.g.,“active” and “history”) may be selected within that field, althoughother standards (or no standard) could be employed. An exemplary list offact categories and their component fields is given below. However, itshould be appreciated that this list is provided by way of example only,as aspects of the invention are not limited to any particularorganizational system for facts, fact categories and/or fact components.

Exemplary List of Fact Categories and Component Fields

Category: Problems. Fields: Name, SNOMED status, ICD code.

Category: Medications. Fields: Name, Status, Dose form, Frequency,Measures, RxNorm code, Administration condition, Application duration,Dose route.

Category: Allergies. Fields: Allergen name, Type, Status, SNOMED code,Allergic reaction, Allergen RxNorm.

Category: Social history—Tobacco use. Fields: Name, Substance, Form,Status, Qualifier, Frequency, Duration, Quantity, Unit type, Durationmeasure, Occurrence, SNOMED code, Norm value, Value.

Category: Social history—Alcohol use. Fields: Name, Substance, Form,Status, Qualifier, Frequency, Duration, Quantity, Quantifier, Unit type,Duration measure, Occurrence, SNOMED code, Norm value, Value.

Category: Procedures. Fields: Name, Date, SNOMED code.

Category: Vital signs. Fields: Name, Measure, Unit, Unit type,Date/Time, SNOMED code, Norm value, Value.

In some embodiments, a linkage may be maintained between one or moreclinical facts extracted by CLU engine 104 and the portion(s) of thetext narrative from which they were extracted. As discussed above, sucha portion of the text narrative may consist of a single word or mayinclude multiple words, which may be in a contiguous sequence or may beseparated from each other by one or more intervening words, sentenceboundaries, section boundaries, or the like. For example, fact 312indicating that patient 122 is currently presenting with unspecifiedchest pain may have been extracted by CLU engine 104 from the words“chest pain” in the text narrative. The “active” status of extractedfact 312 may have been determined by CLU engine 104 based on theappearance of the words “chest pain” in the section of the textnarrative with the section heading “Chief complaint”. In someembodiments, CLU engine 104 and/or another processing component may beprogrammed to maintain (e.g., by storing appropriate data) a linkagebetween an extracted fact (e.g., fact 312) and the corresponding textportion (e.g., “chest pain”).

In some embodiments, GUI 200 may be configured to provide visualindicators of the linkage between one or more facts displayed in factpanel 230 and the corresponding portion(s) of the text narrative in textpanel 220 from which they were extracted. In the example depicted inFIG. 3A, the visual indicators are graphical indicators consisting oflines placed under the appropriate portions of the text narrative intext panel 220. Indicator 313 indicates the linkage between fact 312 andthe words “chest pain” in the “Chief complaint” section of the textnarrative; indicator 315 indicates the linkage between fact 314 and thewords “shortness of breath” in the “Chief complaint” section of the textnarrative; indicator 317 indicates the linkage between fact 316 and theword “hypertensive” in the “Medical history” section of the textnarrative; and indicator 319 indicates the linkage between fact 318 andthe word “obese” in the “Medical history” section of the text narrative.However, these are merely examples of one way in which visual indicatorsmay be provided, as other types of visual indicators may be provided.For example, different or additional types of graphical indicators maybe provided, and/or linked text in text panel 220 may be displayed in adistinctive textual style (e.g., font, size, color, formatting, etc.).Aspects of the invention are not limited to any particular type oflinkage indicator.

In some embodiments, when the textual representation of the free-formnarration provided by clinician 120 has been re-formatted and factextraction has been performed with reference to the re-formattedversion, the original version may nevertheless be displayed in textpanel 220, and linkages may be maintained and/or displayed with respectto the original version. For example, in some embodiments, eachextracted clinical fact may be extracted by CLU engine 104 from acorresponding portion of the re-formatted text, but that portion of there-formatted text may have a corresponding portion of the original textof which it is a formatted version. A linkage may therefore bemaintained between that portion of the original text and the extractedfact, despite the fact actually having been extracted from there-formatted text. In some embodiments, providing an indicator of thelinkage between the extracted fact and the original text may allowclinician 120 and/or other user 150 to appreciate how the extracted factis related to what was actually said in the free-form narration.However, other embodiments may maintain linkages between extracted factsand the re-formatted text, as an alternative or in addition to thelinkages between the extracted facts and the original text, as aspectsof the invention are not limited in this respect.

Fact panel 230 scrolled to the bottom of the display as depicted in FIG.3B shows social history fact category 340, procedures fact category 350,and vital signs fact category 360. Within social history fact category340, two clinical facts have been extracted; no facts have beenextracted in procedures fact category 350 and vital signs fact category360. Within social history fact category 340, fact 342 indicates thatpatient 122 currently smokes cigarettes with a frequency of one pack perday. Fact 344 indicates that patient 122 currently occasionally drinksalcohol. Indicator 343 indicates that fact 342 was extracted from thewords “He smokes one pack per day” in the “Social history” section ofthe text narrative; and indicator 345 indicates that fact 344 wasextracted from the words “Drinks occasionally” in the “Social history”section of the text narrative. In some embodiments, visual indicatorssuch as indicators 343 and 345 may be of a different textual and/orgraphical style or of a different indicator type than visual indicatorssuch as indicators 313, 315, 317 and 319, to indicate that theycorrespond to a different fact category. For example, in someembodiments indicators 343 and 345 corresponding to social history factcategory 340 may be displayed in a different color than indicators 313,315, 317 and 319 corresponding to problems fact category 310. In someembodiments, linkages for different individual facts may be displayed indifferent textual and/or graphical styles or indicator types to allowthe user to easily appreciate which fact corresponds to which portion ofthe text narrative. For example, in some embodiments indicator 343 maybe displayed in a different color than indicator 345 because theycorrespond to different facts, even though both correspond to the samefact category.

In some embodiments, GUI 200 may be configured to allow the user toselect one or more of the clinical facts in fact panel 230, and inresponse to the selection, to provide an indication of the portion(s) ofthe text narrative from which those fact(s) were extracted. An exampleis illustrated in FIG. 4. In this example, fact 312 (“unspecified chestpain”) has been selected by the user in fact panel 230, and in responsevisual indicator 420 of the portion of the text narrative from whichfact 312 was extracted (“chest pain”) is provided. Such a user selectionmay be made in any suitable way, as aspects of the invention are notlimited in this respect. Examples include using an input device (e.g.,mouse, keyboard, touchpad, stylus, etc.) to click on or otherwise selectfact 312, hovering the mouse or other input mechanism above or nearby tofact 312, speaking a selection of fact 312 through voice, and/or anyother suitable selection method. Similarly, in some embodiments GUI 200may be configured to visually indicate the corresponding fact in factpanel 230 when the user selects a portion of the text narrative in textpanel 220. In some embodiments, a visual indicator may include a line orother graphical connector between a fact and its corresponding portionof the text narrative. Any visual indicator may be provided in anysuitable form (examples of which are given above) as aspects of theinvention are not limited in this respect. In addition, aspects of theinvention are not limited to visual indicators, as other forms ofindicators may be provided. For example, in response to a user selectionof fact 312, an audio indicator of the text portion “chest pain” may beprovided in some embodiments. In some embodiments, the audio indicatormay be provided by playing the portion of the audio recording of theclinician's spoken dictation comprising the words “chest pain”. In otherembodiments, the audio indicator may be provided by playing an audioversion of the words “chest pain” generated using automatic speechsynthesis. Any suitable form of indicator or technique for providingindicators may be used, as aspects of the invention are not limited inthis respect.

In some embodiments, GUI 200 may be configured to provide any of variousways for the user to make one or more changes to the set of clinicalfacts extracted from the text narrative by CLU engine 104 and displayedin fact panel 230. For example, the user may be allowed to delete a factfrom the set in fact panel 230, e.g., by selecting the “X” optionappearing next to the fact. In some embodiments, the user may be allowedto edit a fact within fact panel 230. In one example, the user may editthe name field of fact 312 by selecting the fact and typing, speaking orotherwise providing a different name for that fact. As depicted in FIG.3A and FIG. 4, in some embodiments the user may edit the status field offact 312 by selecting a different status from the available drop-downmenu, although other techniques for allowing editing of the status fieldare possible. In some embodiments, the user may alternatively oradditionally be allowed to edit a fact by interacting with the textnarrative in text panel 220. For example, the user may add, delete, orchange one or more words in the text narrative, and then the textnarrative may be re-processed by CLU engine 104 to extract an updatedset of clinical facts. In some embodiments, the user may be allowed toselect only a part of the text narrative in text panel 220 (e.g., byhighlighting it), and have CLU engine 104 re-extract facts only fromthat part, without disturbing facts already extracted from other partsof the text narrative.

In some embodiments, GUI 200 may be configured to provide any of variousways for one or more facts to be added as discrete structured dataitems. As depicted in FIG. 4, GUI 200 in some embodiments may beconfigured to provide an add fact button for each fact categoryappearing in fact panel 230; one such add fact button is add fact button430. When the user selects add fact button 430, in some embodiments GUI200 may provide the user with a way to enter information sufficient topopulate one or more fields of a new fact in that fact category, forexample by displaying pop-up window 500 as depicted in FIG. 5. It shouldbe appreciated that this is merely one example, as aspects of theinvention are not limited to the use of pop-up windows or any otherparticular method for adding a fact. In this example, pop-up window 500includes a title bar 510 that indicates the fact category (“Problems”)to which the new fact will be added. Pop-up window 500 also provides anumber of fields 520 in which the user may enter information to definethe new fact to be added. Fields 520 may be implemented in any suitableform, including as text entry boxes, drop-down menus, radio buttonsand/or checkboxes, as aspects of the invention are not limited to anyparticular way of receiving input defining a fact. Finally, pop-upwindow 500 includes add button 530, which the user may select to add thenewly defined fact to the set of facts corresponding to the patientencounter, thus entering the fact as a discrete structured data item.

In some embodiments, GUI 200 may alternatively or additionally beconfigured to allow the user to add a new fact by selecting a (notnecessarily contiguous) portion of the text narrative in text panel 220,and indicating that a new fact should be added based on that portion ofthe text narrative. This may be done in any suitable way. In oneexample, the user may highlight the desired portion of the textnarrative in text panel 220, and right-click on it with a mouse (orperform another suitable input operation), which may cause thedesignated text to be processed and any relevant facts to be extracted.In other embodiments, the right-click or other input operation may causea menu to appear. In some embodiments the menu may include options toadd the new fact under any of the available fact categories, and theuser may select one of the options to indicate which fact category willcorrespond to the new fact. In some embodiments, an input screen such aspop-up window 500 may then be provided, and the name field may bepopulated with the words selected by the user from the text narrative.The user may then have the option to further define the fact through oneor more of the other available fields, and to add the fact to the set ofclinical facts for the patient encounter as described above.

In some embodiments, the set of clinical facts corresponding to thecurrent patient encounter (each of which may have been extracted fromthe text narrative or provided by the user as a discrete structured dataitem) may be added to an existing electronic medical record (such as anEHR) for patient 122, or may be used in generating a new electronicmedical record for patient 122. In some embodiments, clinician 120and/or coding specialist (or other user) 150 may finally approve the setof clinical facts before they are included in any patient record;however, aspects of the present invention are not limited in thisrespect. In some embodiments, when there is a linkage between a fact inthe set and a portion of the text narrative, the linkage may bemaintained when the fact is included in the electronic medical record.In some embodiments, this linkage may be made viewable by simultaneouslydisplaying the fact within the electronic medical record and the textnarrative (or at least the portion of the text narrative from which thefact was extracted), and providing an indication of the linkage in anyof the ways described above. Similarly, extracted facts may be includedin other types of patient records, and linkages between the facts in thepatient records and the portions of text narratives from which they wereextracted may be maintained and indicated in any suitable way.

In some embodiments, one or more clinical facts, either automaticallyextracted from a text narrative by CLU engine 104 or directly entered bya user as discrete structured data items, may be input to fact reviewcomponent 106 for automatic review. In some embodiments, fact reviewcomponent 106 may be programmed to identify opportunities for theclinical documentation of the patient encounter to be improved, and ifany such opportunities are identified, to provide an alert to the user(e.g., clinician 120 or other user 150). Some examples of alerts thatmay be provided are described above. As discussed, any suitable form ofalert, including visual and/or audio alerts, may be used, as aspects ofthe invention are not limited in this respect. In some embodiments, thereview of collected clinical facts to determine opportunities forimproved clinical documentation, and the resulting alerting and/orquerying of the user, may be performed entirely automatically by factreview component 106 or any other suitable component. As used herein,performing a process “automatically” refers to having no required humanparticipation between the input to the process and its correspondingoutput, with all intervening acts performed by machine.

As discussed above, one type of alert that may be provided to a user byfact review component 106 is an alert of a potential opportunity toincrease the specificity of the set of facts ascertained from thepatient encounter. This can be done in any suitable way. In someembodiments, fact review component may be programmed with a set ofdeterministic rules to decide when such a potential opportunity exists.For example, in some embodiments, if a clinical term corresponding toone of the facts is linked to a concept in the formal ontology used byCLU engine 104, and that concept is a parent to one or more specificchild concepts in the ontology, then fact review component 106 maygenerate an alert to query the user as to whether one of the morespecific child concepts can actually be ascertained from the patientencounter. If the user answers in the affirmative, in some embodimentsfact review component 106 may cause the more general fact to be replacedby a more specific version indicated by the user. Similarly, if one ormore concepts in the formal ontology are linked to clinical termsappearing in the set of facts, and if those concepts have relationshipsin the ontology to a fact that could add specificity to the set offacts, and alert and/or query may be generated. As an example, if one ormore conditions documented in the set of facts are known throughontological relationships to be symptoms of a specific diagnosis, insome embodiments fact review component 106 may query clinician 120 orother user 150 as to whether the specific diagnosis may be ascertainedfrom the patient encounter and added to the facts. In some embodiments,as an alternative or in addition to the set of deterministic rules, astatistical model may be used to identify situations in which apotential opportunity to increase the specificity of the set of factsexists.

In another example, one or more of the facts in the set collected(either by fact extraction from a text narrative or by direct entry asone or more discrete structured data items) from the patent encountermay correspond to one or more standard codes used for billing, ordering,evaluating quality of care, or the like. Such standard codes may bespecific to the healthcare institution or may be a standard shared bymultiple institutions. Examples of such standard coding systems include,but are not limited to, ICD codes, CPT (Current Procedural Terminology)codes, E&M (Evaluation and Management) codes, MedDRA (Medical Dictionaryfor Regulatory Activities) codes, SNOMED codes, LOINC (LogicalObservation Identifiers Names and Codes) codes, RxNorm codes, NDC(National Drug Code) codes and RadLex codes. Some such standard codingsystems are hierarchical, in that certain codes within the system aremore specific versions of other codes within the system. For example, inthe ICD-10 coding system, code 120 represents “angina pectoris” (chestpain due to lack of blood and oxygen to the heart muscle). More specificversions of ICD-10 code 120 include 120.0 (“unstable angina”), 120.1(“angina pectoris with documented spasm”), 120.8 (“other forms of anginapectoris”) and 120.9 (“angina pectoris, unspecified”). In someembodiments, if one of the set of facts collected from the patientencounter includes a general-level code such as ICD-10 120, fact reviewcomponent 106 may be programmed to automatically query the user as towhether one of the corresponding specific-level codes could beascertained from the patient encounter instead. In some embodiments,fact review component 106 may present the user with a structured choiceamong the available specific-level codes, and may allow the user tochoose among the available options.

In another example, fact review component 106 may be programmed to alertthe user when a specific fact may be implied by the combination of twoor more facts appearing together in the set of facts collected from thepatient encounter. One example is a set of facts that included adiagnosis of pneumonia as well as a test result indicating thatpseudomonas was found in a sputum culture. Based on a deterministicrule, or a statistical model result, indicating that these two facts incombination may imply a more specific form of pneumonia due to thepresence of an organism, fact review component 106 may query the user asto whether the more specific diagnosis can be ascertained from thepatient encounter.

In some embodiments, an alert that would otherwise be generated from thecurrent patient encounter may be suppressed if there is information inthe patient's medical history that already provides the additionalspecificity. To this end, in some embodiments fact review component 106may have access to a data set of patient history records 160 for patient122, and may query patient history records 160 for such informationprior to generating an alert to the user. For example, if the set offacts from the current patient encounter specifies a condition but doesnot specify whether it is “acute” or “chronic”, but a previous record inpatient history records 160 already specifies that the condition is“chronic”, then fact review component 106 in some embodiments mayautomatically edit the set of facts for the current patient encounter tospecify that the condition is “chronic”, without bothering the user withan alert. However, in some embodiments, even if fact review component106 can obtain such specificity enhancing information automatically, amessage may still be generated to inform the user that the informationis being automatically added, and to allow the user to reject the changeif desired, or to ask the user to approve of the change being made.

In some embodiments, if it is a user 150, and not clinician 122, whoresponds to an alert to increase the specificity of a set of clinicalfacts for a patient encounter, clinician 120 may be prompted to approveany additional information provided by the other user 150 prior tofinally approving the set of facts for the patient encounter. Forexample, in some embodiments user 150 may be a coding specialist who isassigned the task of reviewing and editing the set of clinical facts(which may include billing codes) into a version fit to be incorporatedinto an electronic medical record, patient reports, order forms, orother document types. In such a “back-end” arrangement, the set ofclinical facts settled upon by coding specialist 150 may then in someembodiments be transmitted to clinician 120 to give final approval tothe set of facts. In some other embodiments, coding specialist 150 maynot be required. For example, in a “front-end” arrangement, clinician120 may review and possibly edit the set of clinical facts himself, andfinally approve the set of facts when he is satisfied. This may occurduring the patient encounter in some embodiments, or at some timethereafter (e.g., before clinician 120 finally approves or signs off onthe report) in other embodiments. In either type of arrangement, in someembodiments, processing by fact review component 106 or any othercomponent to provide alerts, decision support, workflow tools or thelike in relation to the set of facts may be performed prior to theclinician's final approval of the set of facts.

In some embodiments, similar processing may be performed by fact reviewcomponent 106 to alert the user when it is determined that anunspecified diagnosis may possibly be ascertained from the patientencounter. As discussed above, examples of such unspecified diagnosesinclude comorbidities of one or more already specified diagnoses, andidentification of one or more already specified diagnoses ascomplications of one or more other specified diagnoses and/orprocedures. For example, if the set of facts collected for the patientencounter specified a diagnosis of pneumonia, and the patient's oxygensaturation is also low, it may be determined that respiratory failure, acomorbidity of pneumonia, may possibly be ascertained from the patientencounter. In such a case, fact review component 106 may generate analert to the user. In some embodiments, such determinations may be madebased on knowledge of best practices, with deterministic rules providingreminders of diagnoses that should be investigated, for best quality ofcare, when other related conditions are present. In other embodiments,such determinations may be made statistically, by inputting thecollected set of facts and/or facts from the patient's medical historyto a statistical model trained on past clinical reports and/or medicalliterature. In this way, patterns of diagnoses that tend to be relatedmay be identified statistically, and alerts may be generated based onthe likelihood that relationships observed in the past will surface inthe current patient encounter. To this end, in some embodiments, factreview component 106 may have access to a data set of medicalliterature/documents 170 (such as past clinical reports from thehealthcare institution and/or from other sources) from which statisticalmodels may be built and updated.

In some embodiments, as discussed above, fact review component 106 maybe programmed to generate an alert when it determines that two or moreof the facts in the set collected from the patient encounter conflictwith each other in some way, or when it determines that one or more ofthe facts in the set conflict with one or more facts in patient historyrecords 160. In some embodiments, fact review component 106 may beprogrammed to automatically generate such alerts based on a known set ofcombinations of facts that have undesirable interactions. For example,an alert may be generated when the set of facts indicate that patient122 has been prescribed a certain medication (drug A) in addition to acertain other medication (drug B) with which it negatively interacts,such that the two medications should not be prescribed together. In someembodiments, the prescriptions of both drug A and drug B may bespecified in the set of facts collected from the current patientencounter, while in other embodiments, the prescription of drug A may bespecified in a fact from the current patient encounter, and theprescription of drug B may be specified in a fact contained in patienthistory records 160. In some embodiments the known set of undesirableinteractions may be represented in a data set locally accessible to factreview component 106, while in other embodiments, fact review component106 may query one or more external data sets (such as those maintainedby pharmacies) to determine whether given facts for patient 122demonstrate any contraindications. In some embodiments, fact reviewcomponent 106 or another suitable processing component may both maintainan internal data set and also query external data sets, for instance forperiodic updates to the internal data set.

In some embodiments, an alert to a conflict may be triggered by acombination of facts, at least one of which does not correspond to amedication. For example, fact review component 106 may generate alertsfor contraindications related to a combination of a medication with anallergy, a medication with a diagnosis, a medication with a patient'sage or gender, a medication with a condition indicated in the patient'shistory, a medical procedure with any of the foregoing characteristics,or any other combination of a planned treatment with another clinicalfact from the current patient encounter or from the patient's historyfor which the planned treatment is known to be contraindicated.

In some embodiments, as discussed above, fact review component 106 maygenerate an alert when it determines that there is an opportunity to addto the clinical documentation of the patient encounter for qualityreview purposes. In some embodiments, fact review component 106 may beprogrammed with a set of deterministic rules to generate automaticalerts in response to certain facts or certain combinations of facts,based on a standard set of quality of care measures. Such a quality ofcare standard may be proprietary and unique to the specific healthcareinstitution or may be a standard that is not institution specific, suchas the PQRI standard or the JCAHO standard. Any suitable quality of carestandard may be used, as aspects of the present invention are notlimited to any particular quality of care standard. In some embodiments,when a collected fact or combination of facts is associated with acertain recommended action on the part of the clinician according to thequality of care standard, an alert may be provided to query the user asto whether the recommended action was performed. For example, if the setof facts specify that patient 122 is a smoker, in some embodiments factreview component 106 may generate an alert to remind clinician 120 tocounsel patient 122 about quitting smoking, and to document thecounseling in the patient record. In another example, if the set offacts specify that patient 122 presented with a heart attack, in someembodiments fact review component 106 may prompt clinician 120 todocument how quickly aspirin was prescribed and/or administered, suchthat proof of compliance with the applicable quality of care standardsmay be documented. In some embodiments, fact review component 106 may beused to generate PQRI score reports, or the like, to send to insurancecompanies as compliance evidence to support reimbursement.

In some embodiments, as discussed above, fact review component 106 oranother suitable component may generate an alert to the user when itdetermines that disambiguation is desired between multiple facts thatcould potentially be extracted from the same portion of the textnarrative. For example, a term in the free-form narration might belinked to two different concepts in the formal ontology used by CLUengine 104, and it might not be likely that both of those concepts wereintended to coexist in the free-form narration. For example, if the textnarrative contains the word “cold”, it may be difficult in some casesfor CLU engine 104 to determine whether clinician 120 intended that wordto mean that patient 122 is cold to the touch, that patient 122 has arunny nose, or that patient 122 has chronic obstructive lung disease(COLD). In such situations, fact review component 106 in someembodiments may provide a structured choice to the user to disambiguatebetween multiple facts tentatively extracted by CLU engine 104. In someembodiments, each of the options provided in the structured choice maycorrespond to one of the multiple tentative facts, and the user maychoose one of the options to specify which fact should actually beextracted from the free-form narration. As discussed above, if the userchoosing among the facts is a person other than clinician 120, such ascoding specialist 150, then in some embodiments clinician 120 may beprompted to approve the user's choice before finally approving the setof facts for the patient encounter. In other embodiments, the user maybe prompted to provide disambiguating information in free-form, ratherthan as a structured choice, as aspects of the invention relating toprompting for disambiguating information are not limited to anyparticular implementation.

In various situations, as discussed above, fact review component 106 maybe programmed to generate an alert including a structured choice among anumber of options corresponding to clinical facts that could possibly beascertained from the patient encounter. Such a structured choice couldinclude a choice among facts that could add specificity to a set ofclinical facts already collected for the patient encounter, a choiceamong facts potentially implied by one or more combinations of factsalready collected for the patient encounter, a choice to disambiguatebetween facts, or any other choice in which one or more structuredoptions are presented to the user, from which the user may choose. Sucha structured choice may be provided in any suitable way, including as avisual and/or audio listing of the options in the structured choice, asaspects of the invention are not limited in this respect. Similarly, theuser's selection of an option from the structured choice may be receivedin any suitable way, including as manual input and/or spoken input, asaspects of the invention are not limited in this respect.

In some embodiments, in response to the user's selection of one of theoptions, fact review component 106 may, for example through use of CLUengine 104, perform an update to the text narrative to make itexplicitly state information corresponding to the selected fact. Forexample, in some embodiments, CLU engine 104 may in a sense workbackward from the selected fact to generate natural language text fromwhich that fact could have been extracted in the forward sense. In someembodiments, the generated text may then be added to the text narrative.When the fact selected by the user through the structured choice is areplacement for or a disambiguation of a fact already extracted from thetext narrative, the generated text may in some embodiments be used toreplace the portion of the text narrative from which the original factwas extracted. In some embodiments, to determine where in the textnarrative to add the generated text when no other text is to bereplaced, CLU engine 104 may again work backward based on how theselected fact would have been extracted from the narrative. For example,in some embodiments CLU engine 104 may identify a section heading in thetext narrative corresponding to the selected fact, and the generatedtext may be added to that section. (e.g., because a selected fact with astatus of “history” would have been extracted from a section with a“history” heading, the corresponding generated text may be added to sucha section in the text narrative.) In other embodiments, generated textmay simply be added to a predetermined location in the text narrative,such as at the beginning or end of the narrative, regardless of thesemantic content of the generated text.

In some embodiments, fact review component 106 may allow the user tospecify a location in the text narrative where the generated text shouldbe inserted, or may allow the user to correct the location initiallydetermined automatically. In some embodiments, CLU engine 104 or anothersuitable component may be used to update the generated text in responseto the user's indication of a new location at which to insert it in thetext narrative. For example, based on whether the user selects alocation that is sentence-initial, sentence-medial or sentence-final, ora location that is its own sentence or is within another sentence, thegenerated text may be adjusted in terms of capitalization, spacing,punctuation, etc., to fit the selected location syntactically. Inanother example, if a selected fact specifies a family history of acertain condition, the gender of one or more pronouns within thegenerated text may be adjusted based on whether the user selects alocation in a sentence about a female relative or about a male relative.As in other situations, if the user selecting an option from astructured choice and/or specifying a location in the text narrative isa person other than clinician 120, in some embodiments clinician 120 maybe prompted to approve the user's selections prior to finally approvingthe set of clinical facts.

It should be appreciated from the foregoing that one embodiment of theinvention is directed to a method 600 for formatting text for clinicalfact extraction, as illustrated in FIG. 6. Method 600 may be performed,for example, by one or more components of a fact review system such asASR engine 102 and/or CLU engine 104, although other implementations arepossible and method 600 is not limited in this respect. Method 600begins at act 610, at which an original text narrative (e.g., a textualrepresentation of a narration of a patient encounter provided by aclinician) may be received. At act 620, the original text may bere-formatted to produce a formatted text narrative. At act 630, one ormore clinical facts may be extracted from the formatted text. Method 600ends at act 640, at which a linkage between at least one of the clinicalfacts and a corresponding portion of the original text may bemaintained.

It should be appreciated from the foregoing that another embodiment ofthe invention is directed to a method 700 for linking extracted clinicalfacts to text, as illustrated in FIG. 7. Method 700 may be performed,for example, by one or more components of a fact review system such asCLU engine 104 and/or fact review component 106, although otherimplementations are possible and method 700 is not limited in thisrespect. Method 700 begins at act 710, at which a plurality of facts maybe extracted from a free-form narration of a patient encounter providedby a clinician. At act 720, a linkage may be maintained between eachfact (or at least two of the facts) and the corresponding portion of thefree-form narration from which it was extracted. Method 700 ends at act730, at which a different indicator may be provided for each fact, toindicate the linkage between that fact and its corresponding portion ofthe free-form narration.

It should be appreciated from the foregoing that another embodiment ofthe invention is directed to a method 800 for analyzing specificity inclinical documentation, as illustrated in FIG. 8. Method 800 may beperformed, for example, by one or more components of a fact reviewsystem such as ASR engine 102, CLU engine 104 and/or fact reviewcomponent 106, although other implementations are possible and method800 is not limited in this respect. Method 800 begins at act 810, atwhich a set of one or more clinical facts may be collected from aclinician's encounter with a patient. At act 820, it may be determinedfrom the set of facts that additional specificity may possibly beascertained from the patient encounter. Method 800 ends at act 830, atwhich a user may be alerted that an additional fact adding specificityto the set of facts may possibly be ascertained from the patientencounter.

It should be appreciated from the foregoing that another embodiment ofthe invention is directed to a method 900 for identifying unspecifieddiagnoses in clinical documentation, as illustrated in FIG. 9. Method900 may be performed, for example, by one or more components of a factreview system such as ASR engine 102, CLU engine 104 and/or fact reviewcomponent 106, although other implementations are possible and method900 is not limited in this respect. Method 900 begins at act 910, atwhich a set of one or more clinical facts may be collected from aclinician's encounter with a patient. At act 920, it may be determinedfrom the set of facts that an unspecified diagnosis may possibly beascertained from the patient encounter. Method 900 ends at act 930, atwhich a user may be alerted that the unspecified diagnosis may possiblybe ascertained from the patient encounter.

It should be appreciated from the foregoing that another embodiment ofthe invention is directed to a method 1000 for updating text in clinicaldocumentation, as illustrated in FIG. 10. Method 1000 may be performed,for example, by one or more components of a fact review system such asCLU engine 104 and/or fact review component 106, although otherimplementations are possible and method 1000 is not limited in thisrespect. Method 1000 begins at act 1010, at which one or more optionsmay be provided to a user, the one or more options corresponding to oneor more clinical facts that could possibly be ascertained from a patientencounter. At act 1020, a user selection of one of the options may bereceived. Method 1000 ends at act 1030, at which a text narrative (e.g.,a textual representation of a free-form narration of the patientencounter provided by a clinician) may be updated to identify the factcorresponding to the selected option as having been ascertained from thepatient encounter.

A clinical fact review system in accordance with the techniquesdescribed herein may take any suitable form, as aspects of the presentinvention are not limited in this respect. An illustrativeimplementation of a computer system 1100 that may be used in connectionwith some embodiments of the present invention is shown in FIG. 11. Oneor more computer systems such as computer system 1100 may be used toimplement any of the functionality described above. The computer system1100 may include one or more processors 1110 and one or more tangible,non-transitory computer-readable storage media (e.g., memory 1120 andone or more non-volatile storage media 1130, which may be formed of anysuitable non-volatile data storage media). The processor 1110 maycontrol writing data to and reading data from the memory 1120 and thenon-volatile storage device 1130 in any suitable manner, as the aspectsof the present invention described herein are not limited in thisrespect. To perform any of the functionality described herein, theprocessor 1110 may execute one or more instructions stored in one ormore computer-readable storage media (e.g., the memory 1120), which mayserve as tangible, non-transitory computer-readable storage mediastoring instructions for execution by the processor 1110.

The above-described embodiments of the present invention can beimplemented in any of numerous ways. For example, the embodiments may beimplemented using hardware, software or a combination thereof. Whenimplemented in software, the software code can be executed on anysuitable processor or collection of processors, whether provided in asingle computer or distributed among multiple computers. It should beappreciated that any component or collection of components that performthe functions described above can be generically considered as one ormore controllers that control the above-discussed functions. The one ormore controllers can be implemented in numerous ways, such as withdedicated hardware, or with general purpose hardware (e.g., one or moreprocessors) that is programmed using microcode or software to performthe functions recited above.

In this respect, it should be appreciated that one implementation ofembodiments of the present invention comprises at least onecomputer-readable storage medium (e.g., a computer memory, a floppydisk, a compact disk, a magnetic tape, or other tangible, non-transitorycomputer-readable medium) encoded with a computer program (i.e., aplurality of instructions), which, when executed on one or moreprocessors, performs the above-discussed functions of one or moreembodiments of the present invention. The computer-readable storagemedium can be transportable such that the program stored thereon can beloaded onto any computer resource to implement aspects of the presentinvention discussed herein. In addition, it should be appreciated thatthe reference to a computer program which, when executed, performs anyof the above-discussed functions, is not limited to an applicationprogram running on a host computer. Rather, the term computer program isused herein in a generic sense to reference any type of computer code(e.g., software or microcode) that can be employed to program one ormore processors to implement above-discussed aspects of the presentinvention.

The phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” “having,” “containing”, “involving”, andvariations thereof, is meant to encompass the items listed thereafterand additional items.

Having described several embodiments of the invention in detail, variousmodifications and improvements will readily occur to those skilled inthe art. Such modifications and improvements are intended to be withinthe spirit and scope of the invention. Accordingly, the foregoingdescription is by way of example only, and is not intended as limiting.The invention is limited only as defined by the following claims and theequivalents thereto.

What is claimed is:
 1. A method comprising: extracting, from a textdocumenting a clinician's encounter with a single patient, a set of oneor more clinical facts representing one or more abstract semanticconcepts, wherein the extracting comprises analyzing the text, via anatural language understanding engine, to identify a set of one or morefeatures of at least a portion of the text, and correlating the set offeatures to the one or more abstract semantic concepts; wherein the oneor more abstract semantic concepts comprise a first diagnosis that theclinician, in the text, indicated that the patient exhibited; whereinthe first diagnosis is a generic diagnosis representing a class of aplurality of more specific subdiagnoses of the first diagnosis; whereinthe method further comprises: analyzing a history record comprising dataindicative of the patient's history to determine an additional factwithout needing to request input of the additional fact; analyzing theset of one or more clinical facts and the additional fact, using atleast one processor, to generate one or more hypotheses for a seconddiagnosis, exhibited by the patient and not documented in the text, thesecond diagnosis being a particular one of the plurality of morespecific subdiagnoses of the first diagnosis that the clinicianindicated that the patient exhibited; and presenting, to a user, thegenerated at least one of the one or more hypotheses.
 2. The method ofclaim 1, wherein the text comprises a free-form narration of the patientencounter provided by the clinician.
 3. The method of claim 1, whereinthe first diagnosis corresponds to a first code in a hierarchical codingsystem, and wherein the one or more hypotheses for the second diagnosisinclude at least one code in the hierarchical coding system that is amore specific version of the first code.
 4. The method of claim 3,wherein the first diagnosis that the clinician indicated that thepatient exhibited corresponds to a first ICD code, and wherein the oneor more hypotheses for the second diagnosis exhibited by the patient andnot documented in the text include at least one child ICD code of thefirst ICD code in an ICD code hierarchy, wherein the first ICD code is aparent ICD code of the at least one child ICD code in the ICD codehierarchy.
 5. The method of claim 1, wherein analyzing the set of factscomprises determining that the second diagnosis is implied by two ormore facts of the set of facts in combination.
 6. The method of claim 1,wherein the alerting comprises presenting one or more optionscorresponding to the one or more hypotheses, and allowing the user tochoose among the one or more options.
 7. Apparatus comprising: at leastone processor; and a memory storing processor-executable instructionsthat, when executed by the at least one processor, perform a methodcomprising: extracting, from a text documenting a clinician's encounterwith a single patient, a set of one or more clinical facts representingone or more abstract semantic concepts, wherein the extracting comprisesanalyzing the text, via a natural language understanding engine, toidentify a set of one or more features of at least a portion of thetext, and correlating the set of features to the one or more abstractsemantic concepts; wherein the one or more abstract semantic conceptscomprise a first diagnosis that the clinician, in the text, indicatedthat the patient exhibited; wherein the first diagnosis is a genericdiagnosis representing a class of a plurality of more specificsubdiagnoses of the first diagnosis; wherein the method furthercomprises: analyzing a history record comprising data indicative of thepatient's history to determine an additional fact without needing torequest input of the additional fact; analyzing the set of one or moreclinical facts and the additional fact to generate one or morehypotheses for a second diagnosis, exhibited by the patient and notdocumented in the text, the second diagnosis being a particular one ofthe plurality of more specific subdiagnoses of the first diagnosis thatthe clinician indicated that the patient exhibited; and presenting, to auser, the generated at least one of the one or more hypotheses.
 8. Theapparatus of claim 7, wherein the text comprises a free-form narrationof the patient encounter provided by the clinician.
 9. The apparatus ofclaim 7, wherein the first diagnosis corresponds to a first code in ahierarchical coding system, and wherein the one or more hypotheses forthe second diagnosis include at least one code in the hierarchicalcoding system that is a more specific version of the first code.
 10. Theapparatus of claim 9, wherein the first diagnosis that the clinicianindicated that the patient exhibited corresponds to a first ICD code,and wherein the one or more hypotheses for the second diagnosisexhibited by the patient and not documented in the text include at leastone child ICD code of the first ICD code in an ICD code hierarchy,wherein the first ICD code is a parent ICD code of the at least onechild ICD code in the ICD code hierarchy.
 11. The apparatus of claim 7,wherein analyzing the set of facts comprises determining that the seconddiagnosis is implied by two or more facts of the set of facts incombination.
 12. The apparatus of claim 7, wherein the alertingcomprises presenting one or more options corresponding to the one ormore hypotheses, and allowing the user to choose among the one or moreoptions.
 13. At least one non-transitory computer-readable storagemedium encoded with a plurality of computer-executable instructionsthat, when executed, perform a method comprising: extracting, from atext documenting a clinician's encounter with a single patient, a set ofone or more clinical facts representing one or more abstract semanticconcepts, wherein the extracting comprises analyzing the text, via anatural language understanding engine, to identify a set of one or morefeatures of at least a portion of the text, and correlating the set offeatures to the one or more abstract semantic concepts; wherein the oneor more abstract semantic concepts comprise a first diagnosis that theclinician, in the text, indicated that the patient exhibited; whereinthe first diagnosis is a generic diagnosis representing a class of aplurality of more specific subdiagnoses of the first diagnosis; whereinthe method further comprises: analyzing a history record comprising dataindicative of the patient's history to determine an additional factwithout needing to request input of the additional fact; analyzing theset of one or more clinical facts and the additional fact to generateone or more hypotheses for a second diagnosis, exhibited by the patientand not documented in the text, the second diagnosis being a particularone of the plurality of more specific subdiagnoses of the firstdiagnosis that the clinician indicated that the patient exhibited; andpresenting, to a user, the generated at least one of the one or morehypotheses.
 14. The at least one non-transitory computer-readablestorage medium of claim 13, wherein the text comprises a free-formnarration of the patient encounter provided by the clinician.
 15. The atleast one non-transitory computer-readable storage medium of claim 13,wherein the first diagnosis corresponds to a first code in ahierarchical coding system, and wherein the one or more hypotheses forsecond diagnosis include at least one code in the hierarchical codingsystem that is a more specific version of the first code.
 16. The atleast one non-transitory computer-readable storage medium of claim 15,wherein the first diagnosis that the clinician indicated that thepatient exhibited corresponds to a first ICD code, and wherein the oneor more hypotheses for the second diagnosis exhibited by the patient andnot documented in the text include at least one child ICD code of thefirst ICD code in an ICD code hierarchy, wherein the first ICD code is aparent ICD code of the at least one child ICD code in the ICD codehierarchy.
 17. The at least one non-transitory computer-readable storagemedium of claim 13, wherein analyzing the set of facts comprisesdetermining that the second diagnosis is implied by two or more facts ofthe set of facts in combination.
 18. The at least one non-transitorycomputer-readable storage medium of claim 13, wherein the alertingcomprises presenting one or more options corresponding to the one ormore hypotheses, and allowing the user to choose among the one or moreoptions.